Previous work has focused primarily on developing ML-based "meta-learners" that can provide point estimates of the conditional

We develop scalable methods for producing conformal Bayesian predictive intervals with finite sample calibration guarantees.

The seminal work of "equalized coverage" proposed an uncertainty-aware fairness notion.

This frameworkcanaccommodate almost anychoice of conformity scores, and in fact many different implementations have already been proposed to address ourproblem. However,itremains unclear howtoimplement aconcrete method fromthis broad family that can lead to the most informative possible prediction intervals.

The first method, explained in Section A1.4.1, consists of directly calibrating a sequence of nested two-sided intervals, as outlined in Section 3.3. The second method, explained in Section A1.4.2, consists of separately calibrating two sequences of lower and upper one-sided confidence intervals, each adopting the significance level α/2 instead of α. Pu j=l ˆϕj(x)amongthefeasible ones with minimal |u l|, whenever the optimization problem does not have a unique solution. Therefore, we can assume without loss of generality that (1) has a unique solution; if that is not the case, we can break the ties at random by adding a little noise to ˆϕ. For any integer T 1, consider an increasing sequence tτ [0,1], for τ {0,...,T}. A nested sequenceofT intervalsindexedbyτ {0,...,T},whichmaybewrittenintheformof St = ˆLm,α(Xm+1;tτ), ˆUm,α(Xm+1;tτ), for appropriate lower and upper endpoints ˆLm,α(Xm+1;tτ) and ˆUm,α(Xm+1;tτ), respectively, is then constructed from (1) as follows.

A flexible conformal inference method is developed to construct confidence intervals for the frequencies of queried objects in very large data sets, based on a much smaller sketch of those data.